Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100)

For comparisons of citation impacts across fields and over time, bibliometricians normalize the observed citation counts with reference to an expected citation value. Percentile-based approaches have been proposed as a non-parametric alternative to parametric central-tendency statistics. Percentiles are based on an ordered set of citation counts in a reference set, whereby the fraction of papers at or below the citation counts of a focal paper is used as an indicator for its relative citation impact in the set. In this study, we pursue two related objectives: (1) although different percentile-based approaches have been developed, an approach is hitherto missing that satisfies a number of criteria such as scaling of the percentile ranks from zero (all other papers perform better) to 100 (all other papers perform worse), and solving the problem with tied citation ranks unambiguously. We introduce a new citation-rank approach having these properties, namely P100; (2) we compare the reliability of P100 empirically with other percentile-based approaches, such as the approaches developed by the SCImago group, the Centre for Science and Technology Studies (CWTS), and Thomson Reuters (InCites), using all papers published in 1980 in Thomson Reuters Web of Science (WoS). How accurately can the different approaches predict the long-term citation impact in 2010 (in year 31) using citation impact measured in previous time windows (years 1–30)? The comparison of the approaches shows that the method used by InCites overestimates citation impact (because of using the highest percentile rank when papers are assigned to more than a single subject category) whereas the SCImago indicator shows higher power in predicting the long-term citation impact on the basis of citation rates in early years. Since the results show a disadvantage in this predictive ability for P100 against the other approaches, there is still room for further improvements.     

Methodik und Ergebnisse einer Experimentreihe über Entwurfsmuster

Zusammenfassung.   Software-Entwurfsmuster sind eine intuitiv einleuchtende Idee, die viele Befürworter hat. Als Forscher dürfen wir die behaupteten Wirkungen von Entwurfsmustern jedoch nicht einfach glauben, sondern müssen sie gründlich prüfen. Dieser Artikel beschreibt die wichtigsten Behauptungen und entwickelt daraus zugeh?rige Forschungsfragen. Wir diskutieren die Methodik für ein Forschungsprogramm, das die Fragen beantworten soll, und skizzieren praktische Beschr?nkungen, unter denen diese Forschung ablaufen mu?. Es folgt eine kurze Beschreibung von drei kontrollierten Experimenten und den Hauptfolgerungen aus ihren Ergebnissen. So sollte man zum Beispiel die Benutzung von Entwurfsmustern in einem Entwurf genau dokumentieren und Entwurfsmuster nicht einsetzen, ohne alternative Entwürfe zu prüfen. Abschlie?end diskutieren wir ein im Rahmen des Forschungsprogramms geplantes viertes Experiment. Der Beitrag dieses Artikels liegt vorrangig in einer Beschreibung und Diskussion wichtiger methodischer Aspekte kontrollierter Experimente in der Softwaretechnik. Eingegangen am 27. Oktober 1998 / Angenommen am 13. M?rz 1999     

Core elements in the process of citing publications: Conceptual overview of the literature

This study provides a conceptual overview of the literature dealing with the process of citing documents (focusing on the literature from the recent decade). It presents theories, which have been proposed for explaining the citation process, and studies having empirically analyzed this process. The overview is referred to as conceptual, because it is structured based on core elements in the citation process: the context of the cited document, processes from selection to citation of documents, and the context of the citing document. The core elements are presented in a schematic representation. The overview can be used to find answers on basic questions about the practice of citing documents. Besides understanding of the process of citing, it delivers basic information for the proper application of citations in research evaluation.     

Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data

Recently, two new indicators (Equalized Mean-based Normalized Proportion Cited, EMNPC; Mean-based Normalized Proportion Cited, MNPC) were proposed which are intended for sparse scientometrics data, e.g., alternative metrics (altmetrics). The indicators compare the proportion of mentioned papers (e.g. on Facebook) of a unit (e.g., a researcher or institution) with the proportion of mentioned papers in the corresponding fields and publication years (the expected values). In this study, we propose a third indicator (Mantel-Haenszel quotient, MHq) belonging to the same indicator family. The MHq is based on the MH analysis – an established method in statistics for the comparison of proportions. We test (using citations and assessments by peers, i.e. F1000Prime recommendations) if the three indicators can distinguish between different quality levels as defined on the basis of the assessments by peers. Thus, we test their convergent validity. We find that the indicator MHq is able to distinguish between the quality levels in most cases while MNPC and EMNPC are not. Since the MHq is shown in this study to be a valid indicator, we apply it to six types of zero-inflated altmetrics data and test whether different altmetrics sources are related to quality. The results for the various altmetrics demonstrate that the relationship between altmetrics (Wikipedia, Facebook, blogs, and news data) and assessments by peers is not as strong as the relationship between citations and assessments by peers. Actually, the relationship between citations and peer assessments is about two to three times stronger than the association between altmetrics and assessments by peers.     

Sport-specific functional movement can simulate aspects of neuromuscular fatigue occurring in team sports

Fatigue protocols have been used over the years to examine muscular exhaustion. As an alternative to approaches in laboratory settings, functional agility protocols claiming to mimic the multifaceted loads of athletic activity have been proposed. This study aimed to examine the effects of a functional agility short-term fatigue protocol (FAST-FP) on neuromuscular function. Twenty-eight healthy sports students (15 males, aged 24.3 ± 2.4 years) completed the FAST-FP, which consists of four components: three counter-movement jumps (90% of individual maximum), a 20-s bout of step-ups, three bodyweight squats and an agility run. Tasks were repeated until the participants no longer achieved the required jump height in two consecutive sets. Outcomes (pre-post) encompassed subjective exhaustion (visual analogue scale [VAS]), maximum isometric voluntary force of the knee extensors (MIVF), reactive strength index (RSI), mean power frequency (MPF, measured using surface electromyography) and maximum knee range of motion (ROM). Post-intervention, VAS (+54 mm) increased significantly, while MIVF (–6.1%), RSI (–10.7%) and MPF (–4.1%) were reduced (p < 0.05). No changes were observed for ROM (p > 0.05). The FAST-FP induces small-to-moderate impairments in neuromuscular function and considerable self-perceived fatigue. Current evidence on exhaustion developing in team sports suggests that this magnitude of fatigue is similar. The protocol might thus be valuable in the evaluation of treatments counteracting post-match fatigue in team sports.